Note! Let the loading of this page come to a complete stop, to make the internal links work.
Disclaimer
Due to lack of time, this page is no longer maintained and should therefore not be used.
/ Michel Sundin |
The purpose of this web page is to help TelORB support personnel assist customers that have problems with their TelORB systems.
The information in these guidelines are valid for TelORB AD9 and AD12.
Other good documentation in the troubleshooting area are the official TelORB O&M documentation and the inofficial TelORB Guidelines.
Your feedback and suggestions for improving these guidelines are very much appreciated; send an e-mail to Michel Sundin.
| Disable/Enable of TelORB processor |
|
|
|
This chapter contains the actual guidelines concerning how to troubleshoot a TelORB system and some of its peripherals.
Use chapter "Contents" above to access the different subchapters.
Questions to ask:
- Which TelORB release is loaded in the system?
(AD9, AD12 or R13)
- Which patch release is loaded in the system?
- Initial load or reload.
- Has the non-loadable TelORB processor just been swaped?
(If so, the MAC addresses are probably not correctly specified.)
- Has the non-loadable TelORB processor previously been successfully reloaded?
(If so, the MAC addresses are probably correctly specified.)
Troubleshooting:
In AD9 big PROM is used and in AD12 mini (small) PROM is used.
Select the appropriate action(s) from the suggestions below.
- If failure during initial load, check the build environment, e.g. epct command
and all related files. The related files should contain the correct MAC
addresses, IDPs, pool allocations, etc.
For AD12 and later (mini PROM):
- For all TelORB processors, check the following processor information, by
using the TelORB Manager/Browser (for the non-loadable processor, this may
not be possible, but the information for the rest of the processors might
give a wink about the information for the the non-loadable processor):
- LM
- Version
- Date
- Verify that the LM and version of plLoadable correspond to that of the
corresponding AD (Adjust AD release and patch number in paths below):
more /ipims_dicos/ipims_dicos10/dists/TelORB-AD12.4/system/runtime/IDP_MD13-Prom/LM_D24-IntelPcSmallProm/MI_attributes
(E.g. Identity: LM/D24, Version: R1G02)
or
more /ipims_dicos/ipims_dicos10/dists/AD12.4-patch/official/patch1/IntelPcSmallProm_LM/MI_attributes
(E.g. Identity: LM/D24, Version: AD12_4p1)
- Verify that the version of plDisk corresponds to that of the
corresponding AD (Adjust TelORB-AD12.4 to valid AD in path below):
more /ipims_dicos/ipims_dicos10/dists/TelORB-AD12.4/system/runtime/IDP_MD13-Prom/LM_MD14-IntelPcProm/MI_attributes
(E.g. Identity: LM/MD14, Version: R1G02)
For AD9 (big PROM):
- Verify that the version of plDisk corresponds to that of the
corresponding AD (Adjust AD release in path below):
more /ipims_dicos/ipims_dicos10/dists/TelORB-AD9.05.3/system/runtime/IDP_MD13-Prom/LM_MD14-IntelPcProm/MI_attributes
(E.g. Identity: LM/MD14, Version: R1B12)
- Check BIOS of the TelORB processor.
- Connect a serial cable with null modem from either a PC or from an IO
processor to the TelORB processor board, log in via a terminal program such as
hyperterminal for PC or tip for UNIX (IO processor), reboot the non-loadable
TelORB processor and check the following:
- PROM identity
- PROM version
- PROM compile time (date)
- Both MAC addresses of the TelORB processor board
- Where the loading stops
Check that the MAC addresses are the same as in the files
/opt/telorb/axe/loadingGroup01_1/networkAddresses (IO processors 1 and 2)
/etc/dhcpd.pxe.conf (IO processors 1 and 2)
epct file (e.g. processor.epct) (Build environment, e.g.
/home/dicostest/tosFiles/to14/ad13.1)
Questions to ask:
If initial load, make sure that the involved applications are built towards the
equivalent TelORB release that is being loaded in the target machine.
Troubleshooting:
Questions to ask:
- Has the ethernet switch settings been changed?
- Is a Knuerr connected to any of the ethernet switches?
Troubleshooting:
- Verify that the ethernet switch ports that are connected to the IO processors
(most often ports 11 and 12). Check both ethernet switches. "reqPortMode" and
"actualPortMode" for ports 11 and 12 should be set to "100half".
- Temporarily disconnect the Knuerr and try again. Be sure to connect the Knuerr
after verification.
The following error-indicating printouts could be received during loading of a
TelORB system, but may as well be received when the system is up and running.
Example of printouts in the TelORB processor console:
DUCK-server: Heartbeat timeout...
DUCK-server: Connection to Traffic System was shut down...
DUCK-server: Trying to reestablish...
:
DUCK-server: Connection to Traffic System established...
:
Jan 23 09:54:10 io1-hme0 FileServerInterpreter[322]: handleException received i1
Jan 23 09:54:10 io1-hme0 FileServerInterpreter[322]: handleException received i1
Explanation:
- The load on the TelORB IO processor(s) is probably too high.
Solution:
- Check if the load on the TelORB IO processors is too high by entering the
`top' command:
top (to quit `top', enter `q')
- Check how much CPU capacity that is idle by checking the "CPU states" in
the `top' printout.
- Check which processes that take the most CPU capacity by checking the
"%CPU" column (the process that takes the most CPU capacity is displayed
at the top and then in a descending order).
- Terminate the processes which take a lot of CPU capacity and which are not
necessary for the TelORB functionality:
ps -ef
kill <PID>
- It may also be worthwhile to check whether the DUCK-server and
FileServerInterpreter are going up and down (which they should not) by
entering the `ps' command a couple of times and checking if the PID of the
actual processes change (which it should not):
ps -ef | grep erver
UID PID PPID C STIME TTY TIME CMD
Actual process: telorb 324 288 0 Jan 11 ? 16:10 ./FileServerInterpreter.LM_AXG428.AD905p16.solaris -n io1
Actual process: telorb 313 286 0 Jan 11 ? 13:08 ./DUCKServer.LM_MD12.R1E01.solaris -th 134.138.76.101 -th 134.138.76.102 -hbt 1
Guard: telorb 288 1 0 Jan 11 ? 0:00 /bin/csh /opt/telorb/tools/bin/FileServerInterpreter -n io1
Guard: telorb 286 1 0 Jan 11 ? 0:00 /bin/csh /opt/telorb/tools/bin/DUCK-server -th 134.138.76.101 -th 134.138.76.10
The IP addresses in the capsule abortions below are just examples.
Capsule abortion:
Name: DdbnrPpDistributor(134.138.64.107)
Reason: Giving up connect-back to peer node. Received and tried >
134.138.64.107:1065 and 134.138.64.107:1065
Explanation:
The contact between two zones in a network redundant system cannot be
established, due to e.g. bad router configuration.
Solution:
Make sure that the network redundancy links are correctly connected and that
the router(s) inbetween the zones are correctly configured.
Verify with uablrek.
Capsule abortion:
Explanation:
Solution:
Which file contains what?
Regarding the epct file, a customer can either include the epct information in
one epct file or split it into several epct files. The latter is used in the
example epct file names below.
Remember that all IP addresses that are specified below, must be adjusted to
suit the system that you are working on.
o File for setting up the build environment with information about e.g. which
release to build from, as well as information about which epct files to be
included when building the system:
- Envsettings
o File that includes what name the TelORB system will get after the initial
load. The TelORB system name is changed at a software upgrade.
Also, the file contains information about the zone identity if the TelORB
system is part of a network redundant system:
- name.epct (NAME, ZONE)
o Files that include TelORB processor MAC addresses:
- processor.epct (ADDRESS)
Either
- dhcpd.pxe.conf (Only for Pentium III)
or
- 134_138_xx_0 (Only for Pentium I)
- 134_138_yy_0 (Only for Pentium I)
- 134_138_184_0 (Only for Pentium I) (Only if a terminal server is
being used)
- networkAddresses (IO1: /opt/telorb/axe/loadingGroup01_1/)
(IO2: /opt/telorb/axe/loadingGroup01_2/)
(Automatically created at initial load)
o Files that include IP addresses:
- telorbData.epct (DorbIIOPHostName, DUMP_HOST_ID, ETHSW_ADDRESSES)
Either
- dhcpd.pxe.conf (Only for Pentium III)
or
- 134_138_xx_0 (Only for Pentium I)
- 134_138_yy_0 (Only for Pentium I)
- dhcptab (Only for Pentium I)
- 134_138_184_0 (Only for Pentium I) (Only if a terminal server is
being used)
- hosts
- ep_tftpd.conf
o Files that include processor allocations to pools (dependent on the number of
TelORB processors):
- processor.epct (PROCESSOR)
- netred.epct (PROCESSOR)
- demo.epct (PROCESSOR)
- java.epct (PROCESSOR)
- omed.epct (PROCESSOR)
- pools.epct (PROCESSOR)
o Files that are dependent on the number of TelORB processors (also see previous
item, i.e. processor allocations to pools):
Either
- dhcpd.pxe.conf (Only for Pentium III)
(It is OK to include more processors than what
physically exist in the TelORB system. This is
however not valid for the processor.epct file)
or
- 134_138_xx_0 (Only for Pentium I)
- 134_138_yy_0 (Only for Pentium I)
(In the files 134_138_xx_0 and 134_138_yy_0, it is
OK to include more processors than what
physically exist in the TelORB system. This is
however not valid for the processor.epct file)
- start_services (/opt/telorb/tools/bin/STSsw 7 7001 -log &)
(/opt/telorb/tools/bin/DicosSyslogClient -L -m io1
134.138.xx.101-107 > /opt/telorb/syslog &)
- README.txt (For starting consolelogs, located at e.g.
/home/telorb/toX/consolelogs on e.g. a local WS)
o File that includes settings depending on the number of cabinets in the system:
- telorbData.epct (ETHSW_ADDRESSES)
o Files that include information about which software IDPs/IDGs that are being
used:
- telorb.epct (SOFTWARE)
- netred.epct (SOFTWARE)
- demo.epct (SOFTWARE)
- java.epct (SOFTWARE)
- omed.epct (SOFTWARE)
- axmp.epct (SOFTWARE)
- servicePlatform.epct (SOFTWARE)
o Files that include initial data settings:
- telorbData.epct (INITIAL DATA)
- servicePlatform.epct (INITIAL DATA)
o File that includes information about where to put backups and where to reload
from:
- ep_tftpd.conf
|
|
|
|
|
|
http://uabweb.uab.ericsson.se/inside/units/k/kh/d/dv/document/guidelines/troubleshooting.html
